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Abstract.  The  ordering  constraint  along  epipolar  lines  is  a  powerful  constraint  that  has 
been  exploited  by  come  recent  stereomatching  algorithms.  We  formulate  a  ge/ia  allied 
ordering  constraint ,  not  restricted  to  epipolar  lines.  We  prove  several  properties  of  the 
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case,  the  latter  for  a  simplified  but  standard  stereo  geometry.  The  disparity  gradient  limit 
found  in  the  human  stereo  system  may  be  related  to  a  form  of  the  ordering  constraint.  To 
illustrate  our  analysis  we  outline  a  simple  algorithm  that  exploits  the  generalized  ordering 
constraint  for  matching  contours  of  wireframe  objects.  We  also  show  that  the  use  of  the 
generalized  ordering  constraint  implies  several  other  stereo  matching  constraints:  a)  the 
ordering  constraint  along  epipolar  lines,  b)  figural  continuity,  c)  Binford's  cross  product 
constraint,  d)  Mayhew  and  Frisby’s  figural  continuity  constraint.  We  finally  discuss  ways  of 
extending  the  algorithm  to  arbitrary  3  D  objects.  ^ 
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Figure  1  The  geometry  of  the  epipolar  lines. The  plane  defined  by  the  two  foci,  /■’,  and 
h\.  and  a  point  /'  cuts  the  two  image  planes  in  the  corresponding  epipolar  lines  l ,  and  I2. 
As  we  vary  the  position  of  /’  we  produce  a  1  -parameter  family  of  epipolar  lines  on  each 
screen.  The  lines  are  not  parallel  to  one  another,  and  it  can  be  seen  that  the  epipolar  lines 
on  screen  1  all  intersect  at  the  point  where  the  line  /’,  1'2  hits  screen  1. 


1.  Introduction 

The  problem  of  stereo  matching  is  ill  posed  and  underdetermined:  constraints  are  needed 
(a)  to  make  the  solution  unique  and  (b)  to  reduce  the  search  problem  among  possible 
matches. 

Marr  and  Poggio  (1979)  originally  identified  two  important  constraints:  (1)  uniqueness,  that 
is.  an  element  in  one  image  in  general  only  corresponds  with  a  single  element  in  the  other 
image,  and  (2)  continuity,  that  is.  stereo  disparity  varies  smoothly  almost  everywhere  in  the 
image  These  constraints  are  powerful  because  they  do  not  depend  on  the  specific  properties 
of  the  scene  but  on  general  properties  of  the  stereo  geometry.  Marr  and  Poggio  (1979) 
proposed  a  stereo  matching  algorithm,  further  developed  by  Grimson  (1981,  1984),  which 
incom  ~r  ites  the  uniqueness  and  continuity  constraints  to  match  zero  crossing  descriptions 
compt,,.  !  at  different  scales  An  ordering  constraint  along  epipolar  lines  has  been  exploited, 
both  implicitly  and  explicitly,  in  several  computer  algorithms  for  stereo  matching,  as  a  special 
instance  of  the  continuity  constraint.  Epipolar  lines  in  the  two  images  are  lines  on  which 
corresponding  points  lie  r he  projections  of  a  point  P  in  space  lie  on  the  plane  defined 
by  P  and  the  two  camera  foci  and.  as  a  consequence,  on  the  two  lines  defined  by  the 
intersection  of  this  plane  with  the  two  image  planes  (see  figure  1). 

This  implies  that  the  matching  problem  can  be  reduced  to  a  one  dimensional  search  if  the 
epipolars  are  known  Most  algorithms  assume  that  the  epipolar  geometry  is  known  (from  a 
known  camera  geometry)  and  that  the  images  are  registered.  Furthermore,  the  ordering  of 
edges  or  other  features  is  usually  preserved  by  stereo  projection  along  epipolar  lines  (that 
is,  if  feature  A  is  to  the  left  of  feature  B  in  the  left  stereo  image,  then  this  spatial  relationship 
is  maintained  in  the  right  stereo  image)  The  ordering  constraint  along  epipolar  lines  follows 
from  the  continuity  of  surfaces  and  the  assumption  of  opacity  As  originally  suggested  by 
Baker  (1982)  the  ordering  constraint  is  violated  in  situations  such  as  figure  2.  Recently, 
Vern  (1984)  has  discussed  the  role  of  the  "forbidden  zone",  where  the  ordering  constraint 
is  violated  (Krol  and  van  de  Grind.  1982  first  introduced  the  notion  of  forbidden  zone).  The 
forbidden  zone  associated  with  each  point  of  the  visible  surface  is  a  set  of  points  in  space 
that  would  have  images  violating  the  ordering  constraint.  If  any  point  in  the  forbidden  zone 


POGGiO 


YUILLE 


B 


Screen  2  Screen  l 

Figure  2  The  forbidden  zone  associated  with  point  A  is  represented  by  the  dashed  region. 
Any  point  in  this  region  such  as  l)  has  projections  which  violate  the  ordering  constraint 
relative  to  A 

would  be  connected  to  the  first  point  by  an  opaque  surface  the  two  images  would  "see" 
uppubiie  sides  of  tiie  surface. 

This  ordering  constraint  can  be  exploited  to  reduce  the  complexity  of  the  search  for  matching 
features,  and  to  eliminate  false  matches.  Interestingly,  there  is  preliminary  evidence  that  the 
ordering  contraint  (and  perhaps  a  stronger  form  of  it)  may  be  implemented  in  the  human 
stereo  system  (Burt  &  Julesz  1980).  The  human  system,  however,  must  often  cope  with 
situations  in  which  the  images  are  not  precisely  registered.  Furthermore,  physical  edges 
are  inherently  two  dimensional  (a  property  that  is  not  exploited  by  the  epipolar  ordering 
constraint).  It  is  therefore  natural  to  ask  whether  the  ordering  constraint  can  be  generalized 
from  epipolar  lines'.  More  precisely,  can  an  ordering  constraint  be  formulated  that  is 
independent  of  the  epipolar  geometry?  The  simplest  use  of  such  a  constraint  would  be 
when  either  the  epipolar  lines  are  unknown  or  their  estimation  is  affected  by  errors. 

In  this  paper  we  show  that  it  is  indeed  possible  to  generalize  the  ordering  constraint.  We 
also  give  an  analytic  definition  of  the  forbidden  zone  and  characterize  its  properties.  We 
show  then  that  the  generalized  ordering  constraint  implies  several  other  constraints  that 
have  been  exploited  in  stereo  matching.  An  algorithm  based  on  this  constraint  has  not  been 
implemented  yet,  but  we  discuss  its  advantages  and  its  limitations. 

The  plan  of  the  paper  is  as  follows  Throughout  the  paper,  unless  specifically  stated,  we 
consider  a  stereo  geometry  in  which  the  two  image  planes  have  the  same  vertical  unit 
vectors.  This  simple  geometry  is  fully  representative  for  most  practical  applications  and  it 
represents  a  good  approximation  for  human  binocular  geometry  (Longuet  Higgins,  1982).  In 
section  2  we  assume  orthographic  projection.  We  prove  a  simple  relationship  between  the 
two  images  of  a  3D  curve  that  leads  to  a  generalization  of  the  standard  ordering  constraint. 
This  relationship  allows  us  to  identify  special  points  in  the  images  that  correspond  uniquely 
to  the  same  physical  point  in  the  object  curve.  The  Generalized  Ordering  Constraint  (GOC) 
implies  several  of  the  specific  constraints  listed  by  Baker  et  al.  (1983)  ,  Mayhew  and  Frisby 
(1981)  (see  their  liyural  continuity  constraint),  and  Ohta  and  Kanade  (1983).  The  ordering 
constraint  breaks  down  when  the  object  curve  enters  the  forbidden  zone.  We  define  the 

1  This  question  was  first  brought  to  our  attention  by  Dr.  V.  Torre 
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forbidden  zone  geometrically  and  algebraically  and  discuss  its  properties.  Section  3  shows 
that  slightly  weaker  results  can  be  derived  for  perspective  projection  (this  includes  the 
ordering  constraint  along  epipolar  lines  as  a  very  special  case).  These  results  again  break 
down  for  the  forbidden  zone,  which  is  more  complex  than  in  the  orthographic  case.  In 
section  3.  we  characterize  fully  the  forbidden  zone  for  the  perspective  case  and  prove  several 
results  about  it.  The  boundaries  of  the  forbidden  zone  correspond  to  the  so  called  Panum 
limiting  case  in  the  psychophysics  of  stereo,  when  one  line  of  sight  just  grazes  the  surface. 
Burt  and  Julesz'  results  suggest  that  human  stereovision  is  limited  to  a  smaller  disparity 
gradient.  In  section  4  we  derive  the  equations  for  the  physical  surfaces  corresponding  to 
this  disparity  gradient  limit.  In  section  5  we  show  how  the  generalized  ordering  constraint 
implies  other  stereo  matching  constraints.  In  section  6  we  outline  an  algorithm  for  stereo 
based  on  matching  contours.  From  a  single  contour  the  algorithm  retrieves  the  viewing 
parameters  and  unambiguously  matches  points  along  the  contour  using  the  generalized 
ordering  constraint.  The  constraint  of  figural  continuity  (Mayhew  and  Frisby,  1981)  follows 
from  the  generalized  ordering  constraint  and  is  implicitly  implemented  in  our  algorithm.  In 
Appendix  2  we  derive  an  explicit  solution  of  the  equations  for  the  viewing  parameters  in  the 
case  of  the  stereo  geometry  suggested  by  Longuet-Higgins,  using  only  two  points. 


2.  The  Ordering  Constraint  and  the  Forbidden  Zone  for  Orthographic 
Projection 


We  assume  a  sicieu-imaging  geometry  of  ihe  type  proposed  by  Lonyuet-Higyins  (i962)  in 
which  the  planes  of  the  horizontal  meridians  of  the  two  eyes  coincide.  Hence,  the  relative 
orientation  of  the  two  eyes  is  defined  by  one  parameter  only.  We  also  assume  orthographic 
projection.  We  show  that  for  any  part  of  the  object  which  does  not  lie  in  the  forbidden  zone, 
there  is  a  simple  relationship  between  the  images  of  the  object  in  the  two  eyes.  This  enables 
us  to  generalize  the  ordering  constraint  (Baker  &  Binford,  1981)  and  to  identify  features  in 
the  images  that  correspond  to  the  same  feature  in  the  object. 

We  define  an  orthonormal  triad  of  vectors  in  each  eye.  By  our  restriction  on  the  geometry, 
the  vertical  direction,  k,  is  the  same  for  each  eye.  The  right  eye  has  vectors  i,  j  and  k, 
where  j  is  normal  to  the  right  image  plane.  Similarly  the  left  eye  has  vectors  i ',  j '  and  k, 
where  j'  is  normal  to  the  left  image  plane  The  convergence  angle  satisfies  j  •  j'  =  cost? 
(see  Figure  3).  We  define  coordinates  A',  Y  and  Z  along  the  i,  j  and  k  axes  respectively. 
Similarly,  we  let  X'  and  Y'  be  coordinates  along  the  i'  and  j'  directions.  Note  that  the 
origins  of  these  systems  of  coordinates  lies  at  the  intersection  of  the  two  image  planes  and 
not  at  the  focal  points  of  the  eyes.  The  following  equations  connect  the  triads  of  the  two 
eyes: 


j  •  j  '  =  COS  1? 
i  •  i '  =r-  COS  t? 

i'  ■  j  s=  —  sin  d 
i  •  j'  =  f  sin  d 


(2.1) 


We  consider  now  a  curve  in  the  right  image  plane,  parameterized  by  a  (which  is  not  the  arc 
length)  and  written  as 


r«(s)  -  A"(s)i  +  Z(s)k  (2.2) 

Under  the  assumption  of  orthographic  projection,  equation  (2.2)  is  the  image  of  an  object 
curve  given  by  r(»)  ,Y(.*)i-^  V'(.i)j  *  Z(.i)k, 
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Figure  3  See  text. 


rL(a)  =  X'(a)\’+Z'(,)k 

where 

X'(s)  =  r(a)  •  i'  =  X(a)cost?  —  V'fsjsiiit? 

Z'(4)=r(4).k  =  Z(-) 

We  partition  the  right  image  curve  into  intervals  for  which  Z  is  a  single  valued  function  of 
X.  These  intervals  are  separated  by  points  at  which  =  0.  For  each  such  interval  X  can 
be  chosen  as  the  parameter  and  we  can  write  the  projection  (in  the  right  image  plane)  as 

tk{X)  =  Xi  +  Z(X)k  (2.5) 

Identification  of  a  in  equation  (2.3)  with  X  yields  (for  the  left  image) 


(2.3) 

(2.4) 


r  =  X'(X)i'  +  Z'(X)k 


(2.6) 


with 


X  '(X)  =  X  cost?  -  y(X)sin  t? 
Z'(X)  =  Z(X) 


Let  us  now  compute  ^  from  equation  (2.4): 


Note  that  (see  Figure  4) 


dX' 

dX 


cos  t? 


dW[), 

dX  ' 


dY  cost? 
dX  ^  sint? 


(2.7) 


(2.8) 


(2.9) 


is  the  condition  that  curve  never  enters  the  "forbidden  zone".  Thus,  if  the  curve  never 
enters  the  forbidden  zone,  i.e.,  it  satisfies  the  condition  (2.9),  then 
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V)  Figure  4  Line  (a)  has  gradient  smaller  than  cold  and  therefore  is  not  in  the  forbidden 

zone.  Line  (b)  has  gradient  larger  than  cold  and  lies  in  the  forbidden  zone.  The  two  views 
are  of  "opposite"  sides. 


dX'  n 

~dX>Q-  (2.10) 

Let  us  consider  now  the  slope  of  the  curve  in  the  two  images  for  each  partition.  The  chain 
rule  yields 

dZ'  _  dZ  dX 

dX'~dXd)T'  (2n) 

Thus,  if  the  curve  is  outside  the  forbidden  zone,  (a)  $£  and  #  have  the  same  sign  (because 
of  equation  (2.10).  Moreover,  (b)  the  zeros  of  correspond  to  zeros  of  Furthermore, 
(cj  equation  (2.10)  gives  an  ordering  constraint  on  matching  corresponding  points  in  the 
two  curves,  since  within  each  partition  ordered  points  on  the  right  image  curve  correspond 
to  points  on  the  right  image  curve  with  the  same  order  (because  of  monotonicity  implied  by 
equation  (2.11)).  7  K  7 


The  Forbidden  Zone  for  Orthographic  Projection 


We  now  define  the  forbidden  zone  and  show  that  it  occurs  precisely  where  the  orderino 
constraints  break  down. 
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Figure  5  Points  A{  and  A2  are  the  images  of  the  physical  point  A  ana  nave  coordinates 
A'(  and  X2,  respectively. 


Suppose  the  two  screens,  screen  1  and  screen  2,  are  at  an  angle  0  to  each  other  as 
illustrated  in  Figures  5  and  6.  Consider  two  points  A  and  A'  with  coordinates  ( A'i,yj )  and 
(.Y  [,>'[)  respectively  relative  to  screen  t.  They  will  be  projected  to  points  Ay  and  A[  on 
screen  1  with  OA,  =  X,  and  ()A\  —  X[.  They  will  be  projected  to  points  A2  and  A2  on 
screen  2  with  OA2  =  X2  —  A'i  cos 0  +  Y\  sin  0  and  OA[  =  X2  =  X[  cost)  +  Y sin  0.  The 
ordering  constraint  will  hold,  provided 

X (  -  A'i  >  0,  if  and  only  ifAT'  -  X2  >  0  (2.12) 

We  have 

X'  -X2  =  (AT;  -  A'J)cost?{l  +  li-^J-tanfl}  (2.13) 

So  a  necessary  and  sufficient  condition  for  (2.12)  to  hold  is  that 

1  +  >  °  (214) 

Note  that  we  have  -n/2  <  0  <  n/2  and  hence  cost?  is  always  positive. 

Equation  (2.14)  can  De  interpreted  as  a  condition  on  the  gradient  m  of  the  straight  line 
joining  A  to  A’.  Substituting  m  for  **3^7  in  (2.14)  gives 

m  >  —  cotO  (2-15) 


which  is  the  same  as  equation  (2.9), 

It  is  easy  to  see  from  Figure  6  that  if  this  condition  is  violated,  screen  1  and  screen  2  will 
see  opposite  sides  of  the  line  joining  A  to  A'.  This  motivates  the  following  definition  of  the 
forbidden  zone: 

•A  point  A'  is  in  the  forbidden  zone  of  a  point  A  if,  and  only  if,  the  two  screens  see 
opposite  sides  of  the  straight  line  joining  A'  to  A 

As  we  have  shown  above,  the  forbidden  zone  is  precisely  the  region  where  the  ordering 
constraint  breaks  down. 
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A 


Figure  6  See  text. 


Observe  from  Figure  6  that  the  definition  of  the  forbidden  zone  is  symmetric,  that  is,  if  A' 
is  in  the  forbidden  zone  of  A,  then  A'  is  in  the  forbidden  zone  of  A.  Note  also  that  the 
forbidden  zone  of  A  depends  only  on  the  position  of  A  and  the  angle  0  between  the  two 
screens.  This  will  not  be  the  case  when  we  consider  perspective  projection.  In  Section  3, 
we  prove  some  results  about  the  forbidden  zone  for  perspective  projection,  which  will  also 
apply  to  orthographic  projection. 


3.  Perspective  Projection 


We  now  consider  the  ordering  constraints  for  perspective  projection. 

The  geometry  is  summarized  in  Figure  7.  The  two  screens  have  unit  normals  at  and  a* 
and  focal  length  m.  The  foci  have  position  vectors  ft,  and  f2  relative  to  the  origin  0.  An 
arbitrary  point  A  in  space  has  position  vector  X  relative  to  0. 

The  centers  of  screen  1  and  screen  2  are  and  r§,  where 


r?  =  ft  -  ma, 
r°  ~  fu  - 


The  equations  of  the  screens  are 


(3.1) 


r  •  a,  —  fi  •  fit  -  m,  for  screen  1 
r  •  a?  —  It  ■  sz2  -  m,  for  screen  2. 

Now  consider  the  projection  of  a  point  X  on  screen  1.  The  line  of  projection  is 


r(X)  =  X  +  X(f,  -  X) 


(3.3) 


From  (3.2),  this  hits  screen  1  at 


X  ■  a,  +  X(f|  -  X)  -fl,  ==  fi  •  a,  -  m 

We  solve  (3.4)  to  obtain 


(3.4) 
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Figure  7  See  text. 


x  1  (»,-X).fll 

and  so  X  is  projected  to  a  point  r,  with 


m(f1-  X)_ 


(3.5) 


(3.6) 


To  find  the  position  X!  of  the  image  of  X  relative  to  the  center  of  the  screen,  rather  than 
relative  to  0,  we  must  subtract  r,  from  r^  Hence, 


X,  =  r,  -  r?  *  — ~ - «(f,  -  X) •  - (fi  -  X)) 

V'l  -  AJ  •  Uj 


Similarly,  the  projection  on  screen  2  is  given  by 


(3.7) 


X*  -  7.—^ - (((fa  -  X)  ■  a2)at  -  (f2  -  X))  (3.8) 

t'a  -  aj  ■ 

Now  we  restrict  ourselves  to  the  geometry  used  by  Lunguet-Higgins  (1982).  In  this  case, 
tne  vectors  a,,u-,fi  and  f.  are  coplanar  and  perpendicular  to  a  vector  k  that  we  take  to 
be  in  the  z-direction.  We  can  choose  the  origin  0  and  the  x-axis  so  that 


f.  -  fa  =  (/,0,0)  (3.9) 

We  define  angles  <pi  and  such  that 

q,  =  (ros0,,8in<^,,O) 

«2  =  (cos  sin  0a ,  0), 


and  two  vectors  r»j  a.  id  o‘  orthogonal  to  o,  and  q2 
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Figure  8  See  text. 


aj  =  (sin  <bi,  —  cos^i.O) 

“1  :  ;  (.Ul) 

a2  =  tsin  (pt ,  -  cos  <p2 , 0) 

The  vectors  aj.  aj  and  k  form  a  right-handed  triad  for  screen  1,  as  do  r2  and  k  for 
screen  2. 

Let  the  components  of  X  be  (X,  Y,  Z).  Its  projection  Xj  on  screen  1  is  given  by  (3.7),  and 
can  be  written 


X,  =(X,a>;+(X|k)k  (3.12) 

Note  Xi  e;,  =  0,  since  Xi  lies  on  the  screen.  Thus  we  can  take  Xt  =  (X(  ■  a\)  and 
Z i  =  (X!  k)  to  be  the  cartesian  coordinates  of  the  image  on  screen  1.  Substituting  from 
(3.11),  (3.9)  and  (3.7)  gives 


/  COS  Ip  1 


z  _  _ tn* _ 

/  cos  <t>\  —  X  cos  <t>\  -  Y  sin  <j>i 

Similarly,  we  have 


*2 

7? 


f  cos  4  X  cos  <j>-i  Y  sin 
—  mZ 

/  cos  <t> 2  +  X  cos  4>t  +  Y  sin  <p2 


(f  sin  <j> j  +  X  sin  <f> 2  —  Y  cos  02) 


(3.H) 


It  is  well-known  that  there  is  an  ordering  constraint  along  epipolar  lines,  and  with  our 
geometry,  one  such  line  occurs  when  /  ().  This  is  illustrated  by  Figure  8,  where  X  and 

X'  lie  in  the  Z  --  0  plane  The  forbidden  zone  of  X  is  the  shaded  region.  Then,  provided 
that  X'  lies  outside  the  forbidden  zone  of  X  the  images  of  X'  on  the  two  screens  will  be 
either  both  to  the  right  of  the  image  of  X,  as  in  Figure  8,  or  both  to  the  left. 
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Screen 


Screen 


!z 


Figure  9  See  text. 

Now  observe  that  both  A',  and  A'*  in  (3.13)  and  (3.14)  are  independent  of  /.  Thus,  if  we 
remove  the  restriction  /  0,  we  obtain  a  similar  ordering  constraint  independent  of  whether 

X  and  X'  lie  on  the  same  epipotar  line.  Intuitively,  it  X  is  to  the  “right"  of  X'  in  space,  then 
the  projections  of  X  will  be  to  the  “right"  of  the  projections  of  X'  on  both  screens. 

These  results  hold  if.  and  only  if,  X'  is  not  in  the  forbidden  zone  of  X.  The  forbidden  zone 
is  complicated  for  perspective  projections,  and  we  discuss  it  in  the  next  subsection. 

One  obvious  application  of  our  generalized  ordering  constraint  concerns  two  contours  on 
the  two  screens.  It  implies  that  there  is  an  ordered  mapping  between  the  two  contours  and 
that  the  points  at  the  end  of  the  contours,  where  —  0,  must  be  identified.  (See  Figure 
9.)  The  ordering  constraint  along  epipotar  lines  arises  as  a  special  case  when  we  take  the 
contours  to  be  the  epipolar  lines. 

The  Forbidden  Zone  for  Perspective  Projection. 


Since  the  A -component  on  the  screen  is  independent  of  the  /-component  in  space,  (3.13) 
and  (3.14).  the  shape  of  the  forbidden  zone  will  be  independent  of  /. 

From  Figure  10,  it  is  clear  that  !<’  is  in  the  forbidden  zone  of  A  if  and  only  if  the  two  screens 
see  opposite  sides  of  the  straight  line  joining  A  to  W. 

The  structure  of  the  forbidden  zone  is  more  complicated  for  perspective  than  for  orthographic 
projection  For  orthographic  projection,  the  angle  subtended  by  A  to  the  two  screens  was 
a  fixed  angle  0,  independent  of  the  position  of  A.  For  perspective,  this  is  no  longer  the 
case,  in  this  section,  we  characterize  the  forbidden  zone  algebraically  and  prove  various 
desirable  properties  about  it. 

Let  the  permissive  zone  of  A  be  the  complement  of  the  forbidden  zone  of  A.  If  H  is  in  the 
permissive  zone  of  A  to  the  left  of  A  we  will  write  If  left  *A.  Similarly,  II  right  *A  means  II 
is  in  the  permissive  zone  of  A  to  the  right  of  A. 

It  is  clear  from  Figure  10  that  ll  left  *A  if  and  only  if  the  angles  Vn  and  V’i*  are  both  positive. 
This  will  be  true  if  and  only  if  the  /-component  of  the  cross-product  of  the  vectors  /•'■■A 
with  h'.H  and  l\A  with  l\li  are  both  positive  (since  the  lines  are  coplanar,  the  sign  of  the 
/-component  is  the  sign  of  sin  <!'■,  and  sint/’t,  respectively). 

The  vectors  l'2A  and  KH  are  written 
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i 

►  _ 

( 


1  «• 

? 


Figure  10  B'  lies  in  the  forbidden  zone  of  A  but  B  does  not. 


FiA  =  (Xl  +  f,Yx,0) 
f2b  =  (X2  +  f,Y2, 0) 

and  their  cross  product  is 

Ft  AX  Fill  =  {(AT,  +  /)r2  -  (X,  +  f)Yt}k 
Thus  the  point  B  is  to  the  left  of  the  line  f \A  if 


(X,  +  f)Y2  -  (X2  +  f)Yt  >  0 


and  to  the  right  if 


(Xt  +  f)Y,  -  (Xt  +  f)Yt  <  0. 
Similarly,  B  is  to  the  left  of  FtA  if 

(X,  -  J)Yt  -  (Xt  -  f)Y t  >  0 

and  to  the  right  if 


(X,  -f)Ya-  (Xt  -  f)Yt  <  0. 

So  we  can  have  B  left  *A  if  and  only  if  both 

(Xt  +  f)Yt  -  (Xt  +  f)Yt  >  0 
(Xt  -  f)Ya  -  (Xt  -  !)Yt  >  0 

and  B  right* A  if  and  only  if  both 

( X t  4  f)Yt  -  (Xt  +  I)Yt  <  0 
(X,  -  f)Y,  -  (Xt  -  !)Yt  <  O' 


YUILLE 


(3.15) 

(3.16) . 

(3.17) 

(3.18) 

(319) 

(3.20) 

(321) 

(3.22) 


i 
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Figure  1  The  line  AB  projects  to  a  line  on  screen  1  and  a  point  on  screen  2. 


Observe  that  if  Y«  >  y,  then  if  B  is  to  the  left  of  FtA  it  is  automatically  to  the  left  of  h\A. 
if  >:  <  ,  then  if  it  is  to  the  left  of  T\A  it  is  automatically  to  the  left  of  t\A.  This  is  an 

alternati  /e  way  of  characterizing  left*  and  right*. 

We  nov  show  that  the  forbidden  zone  has  the  desirable  properties  of  symmetry  and 

transitivity. 

For  svm  netrv.  we  must  show  that  B  left* A  is  equivalent  to  A  right*B.  In  other  words,  if  B 
is  in  me  permissive  zone  of  A  to  the  left,  then  A  is  in  the  permissive  zone  of  B  to  the  right 

and  conversely. 

The  conditions  for  B  left* A  are  given  by  (3.21).  The  condition  for  A  right*B  are,  from  (3.22) 

(*2  +  I)Yi  -  (Jf,  +  f)Y2  <  0 

(A-,  -  m  -  (Xr  -  f)Yt  <0  '  ’ 

*ne  equation  (3  21)  and  (3.23)  are  cleariy  equivalent  and  hence  the  result  is  proved. 

Now  we  consider  transitivity.  We  want  to  show  that  if  C  left*#  and  B  left*/t  then  C  left*4. 

L  •  r;-/i  means  that  F.C  is  to  the  left  of  F2H  and  FXC  is  to  the  left  of  FX1).  From  B  left* A 
we  h.ivr  F-Jt  to  the  left  of  F2A  and  l\ B  is  to  the  left  of  FXA.  So,  F2C  is  to  the  left  of  F2B 
wn.cn  r.  to  the  left  of  F-2A  and  hence,  F2C  is  to  the  left  of  FtA.  Similarly,  FXC  is  to  the  left 
of  / ,  i  Thus  C  left*/t  and  the  result  holds. 


4.  Limits  to  Fusion 


We  tuv  ;•  shown  m  the  preceding  sections  that  when  a  point  B  is  inside  the  forbidden  zone 
of  a  po  nt  /l.  different  sides  of  the  straight  fine  joining  A  to  II  are  projected  to  different 
screens  If  B  lies  on  the  boundary  of  the  forbidden  zone  of  A  then,  as  in  Figure  11,  one 
side  of  he  line  will  be  visible  to  one  screen  but  the  other  screen  will  only  see  a  point.  This 
corresponds  to  Punum's  Limiting  case. 

if  the  point  B  is  moved  to  IF  in  the  permissive  region  of  A  then  the  line  Alt'  can  be  seen  by 
both  sc  een  1  and  screen  2  although  the  projection  to  screen  2  will  be  very  foreshortened, 
it  should  in  principle  be  possible  to  fuse  features  on  the  line  AB'  despite  this  foreshortening. 
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Figure  12  See  text 


In  a  recent  paper,  however,  Burt  and  Julesz  (1980)  claim  that  there  is  a  limit  to  the  relative 
disparities  in  two  screens  that  the  human  visual  system  can  fuse.  If  this  is  so,  then  the 
boundaries  or  the  forbidden  zone  may  be  a  competence  limit  wnile  the  Burt-Julesz  limit  may 
be  a  peiformance  limit.  We  have  so  far  been  assuming  that  the  greatest  slopes  in  space 
that  we  can  see  are  bounded  above  by  the  forbidden  zone  and  Burt-Julesz’  results  could 
tell  us  that  this  limit  cannot  always  be  attained.  To  investigate  this,  we  ask  what  surfaces  in 
space  correspond  to  a  limit  of  disparity  gradient.  We  will  see  that  they  are  straight  lines. 

First,  assume  the  system  is  fixating  at  infinity  so  the  two  screens  are  parallel.  We  illustrate 
this  case  by  figure  (12).  Now  let  A  be  a  point  in  the  center  of  the  visual  field  with  coordinate 
(0,  a).  It  projects  to  points  A,  and  A2  on  the  screens.  These  have  components  (/(l  +  ®),  -m) 
and  (-/(l  +  f),  -m)  respectively.  Let  the  relative  disparity  limit  be  denoted  by  «  (Burt-Julesz 
would  set  a  =  3).  The  relative  disparity  of  two  points  in  space  is  defined  as  the  ratio  of 
the  differences  of  their  images  in  the  two  screens  (see  Fig.  12).  So  the  fusional  limit  would 
occur  for  points  Q  and  P  on  the  two  screens,  where  Q  is  a  distance  X  to  the  left  of  Av  and 
P  is  aX  to  the  left  of  A2.  So  we  have 


22  =  (/(l  +  ^)-X,-m) 

Q£  =  (-/0  +  -)-<*x,-m) 

d 


The  line  QF,  is  given  by 


y  — 


m 

X  -  /i 


(*-/) 


(4.2) 


where  n  =  mf/a,  and  the  line  PF2  is 


v  = 


m 


aX  +  n 


(*  +  /)• 


These  lines  intersect  at  a  point  where 


(4.3) 
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Figure  13  The  screens  fixate  at  A  with  mergence  angle 


V  — 


A(1  +  q) 

2  ft  +  X(a  —  1) 
2m/ 

2 n  +  X(a  —  1 ) 


(4.4) 


Letting  >.  vary,  we  cbtain  a  curve  pacci.ng  through  ,1  which  is  the  greatcat  o!cpe  co 
to  the  Burt-Julesz  limit.  Eliminating  X  from  (4.4)  gives  the  curve 


•  uupu I  lull  I 


3 


ax  =  (^Tt)^(°  ~  V)  (4-5) 

which  is  a  straight  line.  Observe  that  the  gradient  of  the  curve  depends  only  on  a  and  /. 
Setting  a  —  3  gives  a  gradient  of  ff. 

We  also  perform  the  calculation  when  the  visual  system  is  fixating  on  A  as  in  Figure  13. 

It  is  straightforward  to  modify  the  previous  calculation  by  setting 


m'  =  m  cos  ^ 
X '  =  X  cos  <f> 


(4.8) 


Note  that  in  this  case  we  have 


tan^  =  -.  (4.7) 

Since  neither  m  or  x  appear  in  the  final  expression  (4.5),  the  result  is  unaltered  by  the 
change  in  fixation  point.  As  before,  the  gradient  is  proportional  to  y  and  by  (4.7)  it  is  hence 
proportional  to  co »<f>. 


5.  The  generalized  ordering  constraint  and  other  constraints 

Arnold  and  Binford  (1980)  and  Mayhew  and  Frisby  (1981)  suggested  a  figural  continuity 
constraint  for  stereo  matching:  disparity  is  usually  continuous  along  contours.  This  figural 
continuity  constraint  can  be  powerful  in  practical  implementations  as  demonstrated  by 
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Grimson  (1984)  and  Ohta  and  Kanade  (1984).  Our  geometric  model  of  matching  implicitely 
implements  figural  continuity,  because  matching  is  performed  along  continuous  contours 
under  the  constraint  that  the  disparity  gradient  is  not  too  large  (i.e.,  the  contour  does  not 
enter  the  forbidden  zone). 

Binford  (see  Baker  et  al.,  1983)  suggested  surface  occlusion  rules  for  making  explicit  opacity 
and  non -opacity  of  surfaces.  In  particular  the  cross-product  rule  determines  whether  an 
hypothetized  match  between  two  corners  lies  in  the  forbidden  zone.  Again  a  rule  such  as 
this  follows  immediately  from  the  generalized  ordering  constraint. 

6.  Epipolar  lines  as  a  constraint  for  matching 

We  now  proceed  to  show  that  if  the  stereo  geometry  is  known,  perhaps  by  registering 
the  screens  from  the  image  as  in  Appendix  2,  then  there  is  a  simple  relation  between  the 
epipolar  lines  on  the  two  screens,  A  point  on  the  left  screen  will  lie  on  a  unique  epipolar 
line  which  will  therefore  correspond  to  a  unique  epipolar  line  in  the  right  screen. 

We  consider  a  general  point  in  space  (X,  Y,  Z).  On  the  right  screen,  this  projects  to  (Xt  ,Zt) 
by  equation  3. 13, 


Xt  = 


/  cos  <f>i  —  X  cos  <f>i  —  Y  sin  4>i 
mZ 

f  cos  <f>i  —  X  cos  <f>\  —  Y  sin 


(—/sin  4>i  +  X  sin  —  Y  cos  0i) 


(/  sin  4>2  +  X  sin  fa  —  Y  cos  ^j) 


and  on  the  left  screen  to  (XitZ2)  by  (3.14), 


/  cos  *2  +  AT  cos  to  +  Y  sin  4>u u  ”  ' 

Z2  =  - - - -  (6.4) 

/  cos  <f>2  "f-  X  cos  <j>2  +  Y  sin  <frj 

To  derive  our  relation,  we  use  the  four  equations  (6.1)-(6.4)  to  eliminate  the  unknowns  X, 
Y  and  Z ,  which  leaves  us  with  a  single  equation  relating  Xi,  X2,  Z t  and  Zt. 

We  eliminate  Z  by  dividing  (6.2)  by  (6.4)  to  obtain 

£i  _  -{/cos  <f>2  4-  X  cos <t>2  +  Y sin 0a) 

Z2  /  cos^i  —  X  cos^i  —  Y  sin^i 

We  rewrite  (6.1)  and  (6.3)  respectively  as 

X{  —  X i  cos  <t>\  —  msin  ^i} 

4-  Y{~X\  sin  <t>i  +  mcoa<f>i}  (0.6) 

■+  Xtf  coa<f>i  +  m/sin^j  =  0 


•A"!  —  ^2  co-’’  ^2  —  main  ^2} 

+  Y (-.Vj  sin  ^2  ^  m cos  ^2} 

—  X2 1  cos  ^2  ~  mJ  8'n  ^2=0 
We  combine  (6.6)  and  (6.7)  into  a  matrix  equation 
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f—X i  cos0(  —  msin  0( 
X-i  cos  02  —  m  sin  02 


—  X\  sin  0i  +  in  cos0|\/A'\  _  /-ATi/cos0i  -m/»in0, 
— X2  sin  02  +  rn  compjJxyJ  ^X2/cos02  +m/sin02 


(6.8) 


We  invert  the  matrix  to  solve  for  X  and  Y  obtaining-* 


and 


_  f{(XtX,  -  m2)sin(0i  t  02 )  -  m(X,  -+  A^coa^  +  02)} 
{A' |  Xz  +  f/i'-’}  sin(02  —  0i )+  tn(X-i  —  Xi)cos(02  —  0j) 


(6.9) 


o  t  {X\Xi  cos  02  cos  0|  4-  rnX’j 
TV.  v.  7  .„2\  , 


rm  /hn  ! 


We  rewrite  (6.5)  as 


(2ft  COS  0 1  +  ^2  COS  02)/  +  (Zi)  COS  02  -  Zi  COS  01 )  A" 

+  (Z2  sin  02  —  /?i  sin  <pi)Y  =  0. 

and  substitute  (6.9)  and  (6.10)  into  (6.11).  After  some  manipulation,  this  simplifies  and  we 
obtain  the  result 


/?i{msin02  +  Xi  cos 02 )  =  ^2{msin0i  +  Xj  cos0t}  (®-f2) 

Thus  wp  spp  that  if  we  know  the  position  on  one  screen,  for  example  {Xi,Z J,  we  find 
that  the  other  position  on  the  other  screen,  (Xz,  Z>),  must  lie  on  a  straight  line.  This  is,  of 
course,  the  standard  epipolar  line  constraint.  It  is  interesting  to  see  that  in  our  coordinate 
system,  it  can  be  written  in  such  a  simple  form.  This  makes  it  straightforward  to  implement 
it  in  an  algorithm. 

7.  Outline  of  a  contour  based  algorithm 

A  simple  example  of  an  algorithm  which  indicates  how  the  generalized  ordering  constraint 
can  be  exploited  is  as  follows. 

Instead  of  using  the  epipolar  lines  to  impose  an  ordering  constraint,  we  use  them  as  a 
consistency  constraint.  In  conjunction  with  the  ordering  constraint  along  the  contour,  they 
determine  stereo  matching.  This  avoids  wasteful  scanning  of  all  epipolar  lines  and  ensures 
that  computation  is  done  only  at  places  where  it  is  necessary.  It  also  automatically  enforces 
figural  continuity.  There  are  clearly  interesting  effects  when  the  contour  we  are  matching 
runs  along  an  epipolar  line  for  some  distance.  Psychophysical  experiments  (Buelthoff  and 
Poggio,  pers.  comm.)  suggest  that  the  human  visual  system  also  has  problems  in  such 
cases. 

For  a  single  contour  the  generalized  ordering  constraint  tells  us  that  the  leftmost  point 
and  the  rightmost  point  in  the  two  images  must  correspond  (see  figure  9).  These  points  can 
be  used  to  "register"  the  viewing  system,  using  the  formulae  in  Appendix  2,  and  hence  to 
determine  implicitely  the  epipolar  lines.  Starting  at  one  of  these  two  points  we  move  along 
the  contours  using  the  epipolar  line  constraint  (6.12)  to  determine  which  points  are  to  be 
matched.  If  more  than  one  contour  is  present  the  generalized  ordering  constraint  is  used 
to  decide  which  contours  in  the  two  images  correspond  (contours  are  also  "ordered"  in  a 
similar  way  as  points  along  a  single  contour  are).  In  this  case  there  will  be  many  points  in 
the  two  images  which  are  known  to  correspond  (two  for  each  contour)  and  the  registration 
process  will  hence  be  more  robust  (Mayhew  and  Longuet  Higgins,  1982).  An  algorithm  of 
this  type  is  currently  under  development  with  E.  Tiffany. 
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8.  Discussion 

A  few  remarks  about  photometric  effects  and  perspective  invariants  are  relevant  at 
this  stage.  We  have  assumed  that  the  contours  in  the  image  being  matched  correspond 
to  a  physical  structure,  such  as  a  wire  or  a  silhouette,  in  the  object  being  viewed.  If 
zero-crossings  of  the  Laplacian  of  a  Gaussian  are  used  as  the  matching  primitives  we  must 
be  sure  they  correspond  to  a  precise  physical  location  in  the  object  being  viewed.  It  is 
encouraging  that  results  by  Vuille  (1984)  suggest  that  many  zero-crossings  may  be  due 
to  significant  changes  in  the  geometry  of  the  object  which  are  almost  independent  of  the 
viewing  positions.  As  it  stands  now,  the  theory  presented  here  is  only  valid  for  wire  frame 
objects.  Occluding  contours,  though  theoretically  "wrong",  may  however  be  often  used  in 
small  angle  stereo  (Grimson,  1981).  The  main  challenge  is  to  extend  the  matching  scheme 
presented  here  to  solid  and  textured  surfaces.  Several  possibilities  can  be  considered:  a) 
fingerprints  representations  (Yuille  and  Poggio,  1983)  may  provide  specific  features  to  be 
used  in  the  matching  process:  b)  several  functional  measurements  of  the  two  images  (Krass, 
1984)  along  the  contour  may  be  used  to  perform  the  matching. 

The  generalized  ordering  constraint  analysis  is  done  under  the  assumption  of  a  three- 
dimensional  curve  in  space.  The  analysis  may  be  applied  to  occluding  contours,  but  we 
need  estimates  of  the  errors.  We  also  need  to  identify  which  contours  are  occluding 
contours.  One  possibility  is  to  use  fingerprints  (Yuille  and  Poggio,  1983).  The  idea  would 
then  be  to  identify  separately  zero  crossings  corresponding  to  occluding  contours,  zero 
crossings  corresponding  to  step  edges  and  zero  crossings  due  to  texture.  Some  of  these 
zero  crossings  could  then  be  used  for  a  matching  scheme  based  on  the  generalized  ordering 
constraint.  Interpolation  of  the  surface  between  matched  zero  crossing  could  be  performed 
under  the  constraint  of  matching  measurements  provided  by  a  scheme  similar  to  Kass’ 
(1984).  His  scheme  may  be  reformulated  to  exploit  a  form  of  ordering  constraint  for  speeding 
up  the  search.  The  main  problem  for  a  practical  algorithm  is  obviously  the  stability  of  the 
contours  to  be  matched  between  the  two  images.  Geometric  and  photometric  distortions 
are  likely  to  present  a  hard  problem. 

It  is  also  possible  to  use  perspective  invariants  to  help  match  two  contours  which 
arise  from  the  same  physical  location.  Points  on  the  two  contours  which  correspond  to  the 
same  invariant  can  be  matched.  A  number  of  perspective  invariants  or  "semi-invariants" 
are  discussed  by  Yuille  and  Verri  (1984).  For  example,  the  zeros  and  the  discontinuities  of 
curvature  of  a  non -planar  curve  are  preserved  under  perspective  projection  to  the  image 
plane,  although  the  converse  is  not  true.  These  "semi-invariants"  can  be  used  to  find  points 
on  two  contours  which  correspond. 

In  summary,  we  have  shown  that,  with  the  Longuet-Higgins  (1982)  geometry,  a 
generalized  ordering  constraint  holds  provided  the  viewed  object  does  not  enter  the 
forbidden  zone.  This  constraint  is  not  restricted  to  epipolar  lines.  We  characterize  the 
forbidden  zone  for  orthographic  and  perspective  projections  and  discuss  experiments  which 
suggest  that  the  human  visual  system  incorrectly  interprets  objects  in  this  zone.  Results 
by  Burt  and  Julesz  (1980)  suggest  that  limits  to  fusion  occur  before  this  zone  is  entered. 
We  use  the  generalized  ordering  constraint  to  propose  an  algorithm  for  stereo  matching 
along  contours  using  the  epipolar  lines  as  a  consistency  constraint.  We  use  the  generalized 
ordering  constraint  to  determine  points  in  the  images  which  correspond  and  hence  can 
be  used  to  find  the  viewing  parameters.  The  generalized  ordering  constraint  provides  an 
efficient  way  to  match  points  by  scanning  only  along  the  available  contours  instead  of  along 
all  epipolar  lines.  It  also  automatically  imposes  a  figural  continuity  constraint  (Mayhew, 
1983). 
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Appendix  1:  The  Orthographic  Limit  of  Perspective  Projection 

From  equation  (3.7),  we  have  the  perspective  protection  of  a  point  X  on  a  screen  with 
normal  focus  fi  and  focal  length  m  is  given  by  x,t  where: 

Xl  =  IT  "  X)  -  "  x)  (/I  f  !) 

The  term  inside  the  brackets  corresponds  to  the  orthographic  projection  of  (fi  -  X)  on 
screen  1.  It  is  scaled  by  a  term  (ft  -  X)  a,.  Provided  we  stay  in  a  region  of  the  image 
where  this  term  only  varies  a  little,  orthographic  projection  will  be  a  good  approximation 
and  the  scaling  will  be  constant.  This  will  normally  be  the  case  when  (f  -  X)  and  a,  are 
almost  parallel,  since  we  have 

(f- X)  a,  =  |f- Xjcosr  (A.  1.2) 

where  r  is  the  angle  between  (f  -  X)  and  m-  For  r  near  zero,  cosr  is  approximated  by 
1  -  r2/2  and  will  be  insensitive  to  small  changes  in  r. 

How  good  the  orthographic  approximation  is  in  general  will  depend  on  the  gradient  of 
the  surface  being  projected  not  being  too  large. 


Appendix  2:  Solving  for  the  Viewing  Angles 

The  projection  X,  of  a  point  X  on  screen  1  is  given  by  equation  (3.7),  and  we  write  it 
as 

X!  =  m«,  -  — — y - (X-f,)  (4.2.1) 

a  —  u  ■  a 

Alternately,  we  can  take  a  point  A  on  screen  1  and  construct  the  line  joining  it  to  the 
focus  (A)-  The  point  A  has  position  vector  Zi  relative  to  the  origin  X0  of  screen  1.  From 
Figure  (1)  and  Figure  (8)we  see  that 

X„  =  ft  -  ma,  (A. 2.2) 

The  gradient  of  the  line  from  A  to  A  is  ma,  -  Z,,  and  we  can  write  the  line  as 

r(X)  —  fi  -  ma,  ■+  Zi  +  X{mQ,  -  Z,}  (4.2.3) 

We  rewrite  (A. 2. 3)  as 

r(X)  =  f,  +  (X  -  l){ma,  -  Zi}  (A.2A) 

and  we  obtain  a  similar  equation  for  screen  2: 

r(n)  —  f.  +  (/t  _  l){ma.,  -  Z2}  (yt .2.5) 
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These  lines  intersect  at  a  point  P  where 


f,  4-  (X  -  l){mai  -  Z i }  f2  +  [f*  ~  l){w*S2  ~  Z2}  (/t.2.6) 

Now  in  our  geometry  there  is  a  fixed  (vertical)  axis  k  such  that 

fll  k  ==  s2  k  =  f,  k  =  f2  •  k  =  0  (-4.2.7) 

p ,  lies  on  screen  1  so  Z,  •  ai  =  0  and  so  we  can  define  coordinates  Zi(=  Zt  k)  and 
Xx  of  Zt  by 


Z,  =  Z,k  + Y,(k  X  a,)  (-4.2.8) 

Similarly,  for  Z2  we  have 

Z2  =  Z2k  +  X2(k  x  a2)  (-4.2.9) 


Taking  the  dot  product  of  (A. 2.6)  with  k  yields 


0  =  -(/i  -  [)Z2  +  (X  -  l)Zj  (A.2.10) 

For  convenience  we  replace  X  and  ft  with  (  and  v  where 


f  =  X-1 
v  =  n  ~  l 

Then  we  rewrite  (A. 2.6)  using  (A.2.10)  as 

fi  -  U  —  u{tna2  -  Y2(k  X  fl2)) 
-  f(r»Si  ~  Yt(k  X  at)) 


and  equation  (A.2.10)  as 


£ _  Zt 

v  Z\ 


Substituting  for  f  from  (A.2.13)  into  (A.2.12)  gives 


(A. 2.11) 


(>4.2.12) 


(A.2.13) 


fi  -  f2  -  i/{(ma2  -  Y2(k  X  a2))  -  ^(mfl|  -  Xi(k  X  aj)}  (A. 2. 14) 

Now  we  assume  a  second  point  is  also  known  to  correspond.  This  has  coefficients 
Y ,  on  screen  1  and  Z2,  Y2  on  screen  2.  Repeating  the  argument  leading  up  to  (A. 2.14) 
gives  the  equation 


f»  “  f2  ~  r((m-*2  —  Y2(k  X  a2))  —  —  (m«j  -  Y((k  X  a,))}  (A. 2. 15) 

for  some  constant  r. 

Define  21  =  f,  -  f2  and  the  angles  0  and  ^  as  in  figure  (14).  Then  taking  the  dot 
product  of  (A.2.15)  with  »,  and  -t2  yields 


C 
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Figure  14  See  text 


1  7 

—2/  co s(0  +  <j>)  =  m  cos  0  —  X%  sin  0  —  ~ - 


1  Zt 

-2/  cos  ^  —  m  —  ^-(m  cos  0  +  sin  0) 

IS  41 


Similarly  from  (A.2.15)  we  can  obtain 


1  7 

-2/cos(0  4-  0)  =  m  cos  0  —  3T2  sin  0 - - 

r 


1  ~Z* 

-2/  cos  4>  —  m  —  (m  cos  0  +  Xi  sin  0) 
r  7i 


Dividing  (A.2.16)  by  (A.2.18)  yields 


r  m  cos  0  —  X2  sin  0  —  m 
^  m  cos  0 -- X2  sin  0  —  ^*-m 


Similarly,  dividing  (A. 2. 17)  by  (A. 2. 19)  yields 

r  m  —  f-^(m  cos  5  +  Xi  sin  0) 


u  m  -  ?*(m  cos  0  +  X  i  sin  0) 

Hence  we  can  combine  (A, 2. 20)  with  (A. 2. 21)  to  yield  an  equation  in  0  only. 

Z  X 

(mcos  0  —  Xj  sin  0  —  ~  m)(m  — — (mcos0  +  X\  sin  0))  = 

"i  1 1 

y  y 

(mcos0  —  Xj  sin  0 - —  m)(tn - ~(mcos(?  +  Xt  sin0)) 
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Set 


-Z2 


B  = 


-Y2 


’  r  /  A.  . 

Then,  after  some  manipulation  we  write  (A, 2. 22)  as 

{m2(A  -  B)  +  (/IX,  X2  -  /iX,X2)}s in2<? 

+  m{(X2  -  X2)  +  A/?(X,  -  Xi)}sint? 

+  rn{B(Xi  -  X2)  r  A(X2  -  X,)}  sill  t?  cos  0  —  0 


(A.2.23) 


(A.2.24) 


We  can  divide  this  equation  by  sin  0  (  sin  0  =  0  corresponds  to  the  special  case  when 
the  eyes  are  fixating  at  infinity)  and  obtain  an  equation  of  form 


Ci  sin  0  +  C2  +  C}  cos  0  =  0 


(A.2. 25) 


where 


Ct  =  m2(A  -B)  +  (AX,X2  -  BXiXt) 

C2  =  m{Y2  —  X2  +  AB(Yi  —  X,)} 

Ci  =  m{£?(X,  —  X2)  +  A(Y2  —  X,)} 

(A. 2. 25)  can  be  written  as  a  quadratic  equation 

(<72  +  Cl)  cos2  0  +  2C2C3  cos  0  +  (Cl  -  C\ )  =  0 


(A.2.2B) 


and  has  solution 


—C2C3  ±  C\\/C\  +  Cj  —  C\ 

“ - mrm - 


(A  .2.27) 


(A.2.28) 


Note  that  {Ci,C2,C3)  •-  (-Cu-Ct,-C3)  as  (A,B)  >-*  ( B,A )  and  (X,,X2, Y,,Y2) 
X),X,1Xi,X2).  So  if  we  have  two  points  in  the  visual  field  which  are  known  to  coincide 
ihen  we  know  that  points  I\  and  T\  on  screen  1  correspond  to  P2  and  l‘2  on  screen  2. 
The  coefficients  (X,,Jfi),(X„7,),  (X2>  Z2)  and  (Y2,J2)  of  Pt,TuPa  and  V2  are  known  and 
nence  from  (A.2.23)  and  (A. 2.26),  C, , C2  and  C3  are  known.  (A.2.28)  gives  two  possible 
solutions  for  0  in  terms  of  CUC2  and  C3.  Let  these  be  0\  and  02.  Dividing  (A.2.16)  by 
[A. 2. 1 7)  yields 


coa( 0  +  <f>)  mcost?  -  X2  sin  0  —  Z2/Zim 
cos  6  m  —  Z2/Zi(mcos0  +  X,  sin  t?) 

Expanding  cos(t?  +  <f>)  =  cost? cos 6  —  sin t?sin <f>  we  can  rewrite  (A. 2.29)  as 

C  i  sin  6  +  Ci  cos  6  =  0 


(A.2.29) 


(  a  ') 


where 


(?!  -  - m  —  Z2/Zi(m  cos  0  X,  sin  0) 

C3  =  —  X2  —  Z2/ Zi(msh\0  —  X|  cost?) 


(A.2.31) 
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There  are  two  possible  values  of  (C’i,G'2,(73)  depending  on  whether  0  =  0,  or  0  --  02. 
(A. 2. 31)  will  give  two  possible  solutions  for  <p ,  for  each  value  of  0: 


cos  <p 


(A. 2  M2) 


Hence  we  have  four  possible  solutions  for  0  and  <j>  which  can  be  formed  directly  from 
(A. 2. 28)  and  (A. 2. 29).  In  the  general  case,  only  one  of  these  solutions  will  be  physically 
reasonable  and  the  others  can  be  discarded. 

We  have  shown  that  if  the  images  on  both  screens  of  points  in  space  are  known  then 
we  can  solve  the  non  linear  equations  directly  for  the  angles  0  and  6  that  specify  the  stereo 
geometry.  These  solutions  are  given  by  (A, 2. 28)  and  (A. 2. 32).  Two  solutions  are  generated 
of  which  only  one  is  physically  reasonable  in  general. 
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